The role of feature visibility constraints in perspective alignment

Perspective Alignment is a novel method of solving backprojection, the well-known problem of computing threedimensional (3D) position and orientation (pose) of a model from two-dimensional (2D) image features. This paper demonstrates that previous back-projection methods can violate the visibility constraint by computing solution poses in which the model occludes features which should be visible. By definition, these visibility errors are associated with incorrect pose solutions. Yet they occur frequently when previous backprojection methods are used in underconstrained situations. We empirically analyze the frequency and consequences of visibility errors in previous back-projection methods. We then show how Perspective Alignment satisfies the visibility constraint during the pose solution process to eliminate these errors. The algorithm has been implemented and used in a realtime model-based object tracking system. We describe the algorithm and results of tracking real objects in real-time. The algorithm also has implications for reducing the combinatorics of image-model feature pairing in model-based recognition.